Humans interact with their surrounding environment through five sensory channels, popularly labeled “sight,” “sound,” “taste,” “smell,” and “touch.” It is our sense of touch which provides us with much of the information necessary to modify and manipulate the world around us. The word haptic refers to something “of or relating to the sense of touch” (Hogan, N., “Impedance Control: An Approach to Manipulation: Part I—Theory,” Journal of Dynamic Systems, Measurement, and Control, vol. 107, pp. 1–7, 1985) conveying information on physical properties such as inertia, friction, compliance, temperature, and roughness. This sense can be divided into two categories: the kinesthetic sense, through which we sense movement or force in muscles and joints; and the tactile sense, through which we sense shapes and textures. This invention is concerned primarily with haptic interactions involving the use of the kinesthetic sense in virtual environments.
Haptics research grew rapidly in the 1990s as researchers and corporations discovered more uses for force feedback technology. One important catalyst in this frenzy of research was the development and commercialization of the PHANTOM family of haptic displays (Massie, T. H. and Salisbury J. K., “The Phantom Haptic Interface: A Device for Probing Virtual Objects,” Proc. ASME International Mechanical Engineering Congress and Exhibition, Chicago, pp. 295–302, 1994). At the University of Washington, a small, portable, desk-top system was developed for interaction with three degree-of-freedom environments, the Pen Based Force Display (PBFD) (Buttolo, P. and Hannaford, B., “Pen Based Force Display for Precision Manipulation of Virtual Environments,” Proc. IEEE Virtual Reality Annual Int. Symposium, Raleigh, N.C., pp. 217–225, 1995). Another compact master device, developed at IBM and Carnegie Mellon University for teleoperation, departs from more conventional designs by suspending the handle using magnetic levitation (Hollis, R. L., Salcudean, S. E., and Allan, A. P., “A Six-Degree-of-Freedom Magnetically Levitated Variable Compliance Fine-Motion Wrist: Design, Modeling, and Control,” IEEE Trans. Robotics and Automation, vol. 7, no. 3, pp. 320–332, 1991). Subsequent versions of the device have been used at the University of British Columbia as haptic interfaces to virtual environments (Salcudean, S. E., Wong, N. M., and Hollis, R. L., “Design and Control of a Force-Reacting Teleoperation System with Magnetically Levitated Master and Wrist,” IEEE Trans. Robotics and Automation, vol. 11, no. 6, pp. 844–858, 1995; Salcudean, S. E., and Vlaar, T. D., “On the Emulation of Stiff Walls and Static Friction with a Magnetically Levitated Input-Output Device,” Trans. ASNE, Journal of Dynamic Systems, Measurement, and Control, vol. 119, no. 1, pp. 127–132, 1997).
While small desk-top manipulanda, such as those mentioned above, are useful for many applications, others demand a much larger workspace and higher force output. The University of Washington High-Bandwidth Force Display (HBFD)(Moreyra, M. R., “Design of a Planar High Bandwidth Force Display with Force Sensing,” M. S. Thesis, University of Washington, Department of Electrical Engineering, Seattle, 1996) and Excalibur (Adams, R. J., Moreyra, M. R., Hannaford, B., “Excalibur—A Three Axis Force Display,” Proc. ASME International Mechanical Engineering Congress and Exhibition, Nashville, Tenn., 1999) provide large workspace and high forces through direct drive motors and a novel steel cable transmission. In contrast to manipulanda which provide forces only to the user's hand or fingers, exoskeletal systems may generate sensations affecting an entire limb. One such device was developed at the Scuola Superiore Sant' Anna in Pisa, Italy (Bergamasco, M., et al., “An Arm Exoskeleton System for Teleoperation and Virtual Environments Applications,” Proc. IEEE Int. Conf. Robotics and Automation, Los Alamitos, Calif., pp. 1449–54, 1994). The Sarcos Dexterous Arm Master is another exoskeleton-like system with contact points at the forearm and upper arm of the user and is being used to provide force feedback in CAD applications (Maekawa, H., and Hollerbach, J. M., “Haptic Display for Object Grasping and Manipulating in Virtual Environment,” Proc. IEEE Int. Conf. Robotics and Automation, Leuven, Belgium, pp. 2566–73, 1998).
The early history of force feedback technology, dominated by research applications, changed in 1996 when CH Products released the first consumer level haptic display, the Force FX joystick. Microsoft entered the market in 1997 with the Sidewinder Force Feedback Pro joystick. By working with the Immersion Corporation to integrate force feedback technology into the industry standard DirectX API, they gained rapid acceptance among programmers. Logitech joined the mix in 1998 with the WingMan Force joystick, a cable-driven device which raised the bar for fidelity in consumer haptic systems. The consumer market penetration of haptics continues with the arrival of numerous force feedback steering wheels for use with racing simulators.
Virtual environments of interest are always non-linear and the dynamic properties of a human operator are always involved. These factors make it difficult to analyze haptic systems in terms of known parameters and linear control theory. One discipline which becomes more important with the rapid growth of haptics is control engineering. The control engineer is concerned with ensuring the haptic system, including the haptic display, the application software, and the human operator, remains stable while creating a compelling sense of haptic presence.
Early efforts for control in haptics can be found in the adaptation of two-port network theory to the analysis of teleoperators and passivity criteria to design control gains for a master-slave manipulator (Raju, G. J., Verghese G. C., and Sheridan T. B., “Design Issues in 2-port Network Models of Bilateral Remote Manipulation,” Proc. IEEE Int. Conf. Robotics and Automation, Scottsdale, Ariz., pp. 1316–21, 1989; Hannaford, B., “A Design Framework for Teleoperators with Kinesthetic Feedback,” IEEE Trans. Robotics and Automation, vol. 5, no. 4, pp. 426–434, 1989; Hannaford, B., “Stability and Performance Trade Offs in Bi-Lateral Telemanipulation,” Proc. IEEE Int. Conf. Robotics and Automation, Scottsdale, Ariz., pp. 1764–7, 1989; Anderson, R. J. and Spong, M. W., “Bilateral Control of Teleoperators with Time Delay,” IEEE Trans. Automatic Control, vol. 34, no. 5, pp. 494–501, 1989). In the mid-1980s, Neville Hogan discovered that while the neuromuscular system is internally complex, it exhibits externally simple, spring-like behavior (Hogan, N., “Multivariable Mechanics of the Neuromuscular System,” Proc. IEEE Annual Conference of the Engineering in Medicine and Biology Society, pp. 594–98, 1986). The significance of this result is that the human arm can be assumed stable when coupled to any external system which is itself passive. Human arm impedance can therefore be considered as passive for the purposes of studying system stability.
Faster and cheaper computer growth has enabled haptic feedback with increasingly complex virtual reality simulations, but the stability of haptic feedback now depends on the intricate interactions taking place in the virtual world, adding complex geometry to the factors affecting stability. One of the most significant problems in haptic interface design is to create a control system which simultaneously is stable (i.e., does not exhibit vibration or divergent behavior) and gives high fidelity under any operating conditions and for any virtual environment parameters. This presents a classic engineering trade off since realism of the haptic interface (for example in terms of stiffness of “hard” objects) must often be reduced in order to guarantee totally stable operation. In 1995, Colgate, Stanley, and Brown proposed the introduction of a “virtual coupling” between the haptic display and the virtual environment to deal with this problem (Colgate, J. E., Stanley, M. C., and Brown, J. M., “Issues in the Haptic Display of Tool Use,” Proc. IEEE RSJ Int. Conf. Intelligent Robots and Systems, Pittsburgh, Pa., pp. 140–145, 1995). J. Michael Brown explored conditions under which the virtual coupling parameters guaranteed a passive interface to the human operator. He developed design criteria for an arbitrary discrete-time passive environment (Brown, J. M. and Colgate, J. E., “Passive Implementation of Multibody Simulations for Haptic Display,” Proc. ASME International Mechanical Engineering Congress and Exhibition, Dallas, Tex., pp. 85–92, 1997) and for a non-passive virtual mass simulation (Brown, J. M. and Colgate, J. E., “Minimum Mass for Haptic Display Simulations,” Proc. ASME International Mechanical Engineering Congress and Exhibition, Anaheim, Calif., pp. 249–56, 1998). Zilles and Salisbury from the MIT Artificial Intelligence Laboratory presented their own technique for stable haptic rendering of complex virtual objects (Zilles, C. B. and Salisbury, J. K., “A Constraint-based God-object Method for Haptic Display,” Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, Pittsburgh, Pa., pp. 146–151, 1995). Their approach was to servo the haptic display to an artificial “god-object” which conformed to the virtual environment. MIT's god-object was actually a special case of Northwestern's virtual coupling, for point contact with a static virtual environment. The above-mentioned two works were complementary in the sense that Northwestern provided a strong theoretical basis while MIT demonstrated a relatively sophisticated application of the approach.
The virtual coupling is a virtual mechanical system containing a combination of series and parallel elements interposed between the haptic interface and the virtual environment to limit the maximum or minimum impedance presented by the virtual environment in such a way as to guarantee stability. Particulars of virtual coupling design depend the causality of the virtual environment and the haptic device. By causality, we refer to the selection of velocity or force as input and its complement (force or velocity) as output. Possible virtual environment (VE) causalities include impedance based (position/velocity input, force output), admittance based (force input, position/velocity output), or constraint based (position input/position output). While important, the virtual coupling idea considered only one class of haptic displays, “impedance displays” which measure motion and display force. A second class, “admittance displays” which measure force and display motion, was not considered.
The virtual coupling promotes stability by placing an upper limit on the mechanical impedance which can be displayed to the operator. The virtual coupling also inherently distorts the haptic properties built into the virtual environment, reducing environment stiffness and damping in most cases. In the case of an impedance based environment (typical of many implemented systems), a virtual spring and damper in parallel are typically connected in series between the haptic interface and the virtual environment. Stability in this case depends inversely on the stiffness being rendered by the system and the series stiffness has the effect of setting the maximum stiffness. Correct selection of the virtual coupling parameters will allow the highest possible stiffness without introducing instability. The virtual coupling parameters can be set empirically, but a theoretical design procedure is desirable. Because interesting virtual environments are always non-linear and the dynamic properties of a human operator are always involved, it is difficult to analyze haptic systems in terms of known parameters and linear control theory. One fruitful approach is to use the idea of passivity to guarantee stable operation.
The major problem with using passivity for design of haptic interaction systems is that it is overly conservative. Adams et al. derived a method of virtual coupling design from two-port network theory which applied to all causality combinations and was less conservative than passivity based design (Adams, R. J. and Hannaford, B., “Stable Haptic Interaction with Virtual Environments,” IEEE Trans. Robotics and Automation, vol. 15, no. 3, pp. 465–474, 1999; Adams, R. J., Moreyra, M. R., Hannaford, B., “Stability and Performance of Haptic Displays: Theory and Experiments,” Proc. ASME International Mechanical Engineering Congress and Exhibition, Anaheim, Calif., pp. 227–34, 1998; Adams, R. J., Klowden, D., Hannaford, B., “Stable Haptic Interaction using the Excalibur Force Display,” Proc. IEEE Int. Conf. Robotics and Automation, San Francisco, Calif., 2000, pp. 770–775). They were able to derive optimal virtual coupling parameters using a dynamic model of the haptic device and by satisfying Lewellyn's “absolute stability criterion,” an inequality composed of terms in the two-port description of the combined haptic interface and virtual coupling system. This procedure guaranteed a stable and high performance virtual coupling as long as the virtual environment was passive. Miller, Colgate, and Freeman have derived another design procedure which extends the analysis to non-linear environments and extracts a damping parameter to guarantee stable operation (Miller, B. E., Colgate, J. E., and Freeman, R. A., “Passive Implementation for a Class of Static Nonlinear Environments in Haptic Display,” Proc. IEEE Int. Conf. Robotics and Automation, Detroit, Mich., May, 1999, pp. 2937–2942; Miller, B. E., Colgate, J. E., and Freeman, R. A., “Computational Delay and Free Mode Environment Design for Haptic Display,” Proc. ASME Dyn. Syst. Cont. Div., 1999; Miller, B. E., Colgate, J. E., and Freeman, R. A., “Environment Delay in Haptic Systems,” Proc. IEEE Int. Conf. Robotics and Automation, San Francisco, Calif., April, 2000, pp. 2434–2439).
U.S. Patents dealing with haptic interfaces include: U.S. Pat. Nos. 6,111,577; 5,625,576; 6,084,587; and 5,898,599.
All publications referred to herein are incorporated by reference to the extent not inconsistent herewith.